Understand how AppImages work by building one from scratch
What are AppImages?
When you want to install software on a Linux distribution, the easiest way is to use the distribution’s software manager.
The software manager facilitates searching, downloading and installing Package Archives in the particular archive format that the distribution uses.
Debian based distributions use .deb, Redhat based distributions used .rpm.
If you have the package archive, you can manually install it using the following commands:
rpm -U filename.rpm
dpkg -i filename.deb
These packages usually have to be compiled for a specific distribution and are not portable. You cannot easily install a RPM built for Fedora on an Ubuntu system.
There are some software package managers out there such as NIX, HomeBrew which try to produce portable packages. Like the others, these managers also install the package to your system, either in your root folder or in your home directory.
AppImages are a different from of obtaining applications. An AppImage is a single executable file which does not need to be install. Simple download and run.
As an example, we can download and run the application, Obsidian I use to write my posts.
# Download the appimage
wget https://github.com/obsidianmd/obsidian-releases/releases/download/v1.0.3/Obsidian-1.0.3.AppImage
# make it executable
chmod +x Obsidian-1.0.3.AppImage
# Run
./Obsidian-1.0.3.AppImage
Today we are going to understand the technology behind how AppImages work.
FileSystem in UserSpace (FUSE)
Before we dig into AppImages, it’s important to understand a few concepts.
From the Wikipeida article:
Filesystem in USErspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code
What this means is you can mount various filesystems without root privileges. This allows you to change what your filesystem looks like.
What we are going to do is:
Install some needed applications.
sudo apt install squashfs-tools squashfuse
Create a simple FileSystem directory structure with an executable script
# Create an empty directory to work in
mkdir $HOME/fuse-test
cd $HOME/fuse-test
# create a simple filesystem directory structure
mkdir -p fs
# Create as Simple hello world script in the bin folder
cat << EOF > fs/run.sh
#!/bin/bash
echo hello world
EOF
# Make the script executable
chmod +x fs/run.sh
If you have done that correctly, your directory structure should look like this:
.
└── fs
└── run.sh
Compress the FileSystem into a single file using squashfs
mksquash fs myfilesystem.sqsh
If everything worked, you should now have a new file called myfilesystem.sqsh in your working directory.
Mount the file using FUSE
# Create the mount point where it will be mounted
mkdir mnt
# Mount the filesystem
squashfuse myfilesystem.sqsh mnt
If you look at your directory structure, it should now look like this:
.
├── fs
│ └── run.sh
├── mnt
│ └── run.sh
└── myfilesystem.sqsh
SquashFS is a read-only filesystem, meaning you cannot modify it. If you try to delete mnt/bin/run.sh it will give you an error
rm mnt/run.sh
rm: cannot remove 'mnt/run.sh': Function not implemented
But you can still run it
./mnt/run.sh
hello world
Removing the Mount
You can unmount the filesystem using the following command
fusermount -u mnt
Doing so will leave you with an empty mnt directory
.
├── fs
│ └── run.sh
├── mnt
└── myfilesystem.sqsh
How AppImages Work
The basic concept of an AppImage is that the AppImage file you download is actually the filesystem that gets mounted in the /tmp folder and the run.sh script is automatically executed using a start-up script.
The start-up script is responsible for doing the following:
- Mounting the file system
- Executing the run.sh script
- Unmounting the filesystem
The start-up script and the filesystem are concatenated into a single file using the following comment
cat start-up.sh myfilesystem.sqsh > MyApp.AppImage
The single new file we created should look something like this:
+-------------------------------------------------+
| start-up-script.sh | myfilesystem.sqsh |
+-------------------------------------------------+
NOTE: The new file we created is no longer a valid squashfs filesystem so it can no longer be mounted as normal. But what we can do is provide an additional option to the squashfuse command to start reading from a specific byteOffset which we have to figure out.
squashfuse MyApp.AppImage mnt -o --offset=${byteoffset}
The byteOffset is the byte size of the startup script!
Create the Start-Up Script
Create a new file called start.sh and copy the following code into it. The comments in the code should be self explanatory.
#!/bin/bash
###########################################################################
# Get the location of the script
# https://stackoverflow.com/a/246128
##########################################################################
SOURCE=${BASH_SOURCE[0]}
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
DIR=$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )
SOURCE=$(readlink "$SOURCE")
[[ $SOURCE != /* ]] && SOURCE=$DIR/$SOURCE # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR=$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )
##########################################################################
# The AppImage file
IMAGE=$(realpath ${DIR}/${SOURCE})
# figure out the offset by using grep to search the image file for the SECOND
# occurance of a specific string and returning its byte offset.
# The first occurance is in the grep command
offset=$(grep -bas -m2 '#!!NOTHING_BELOW_HERE!!' ${IMAGE} | cut -d ':' -f 1 | tail -n1)
# Add the string length to get the final offset
offset=$((offset+24)) # 23 is the size of the string we are searching for
# Create a temporary mounting point in the /tmp folder
MOUNT_POINT=$(mktemp -d /tmp/.mount_MyApp_XXXX)
# Mount the image and provide it the byte offset
squashfuse ${IMAGE} ${MOUNT_POINT} -o offset=${offset}
# Run the script in the newly mounted directory
${MOUNT_POINT}/run.sh
# Unmount the directory when the script exits
fusermount -u ${MOUNT_POINT}
exit 0
#!!NOTHING_BELOW_HERE!!
Create the final AppImage
Now that you have your SquashFS filesystem and your start.sh script the only thing left to do is combine them and make the new file executable
cat start.sh myfilesystem.sqsh > MyApp.AppImage
chmod +x MyApp.AppImage
You can now run your newly created AppImage
./MyApp.AppImage
hello world
How AppImages Actually Work
What we created is a toy example. It only runs a rudimentary hello world bash script. It is also not an official AppImage format, the above steps only seek to explain the basic technology behind how they work.
AppImages work by copying all the binary files and required shared libraries into a proper directory structure within the filesystem (bin/lib/etc). Then, using the initial run.sh script (they name the file AppRun) to set up required environment variables such as PATH and LD_LIBRARY_PATH and execute the binary.
It also does some more complicated behaviour depending on what the application needs.
You can see what the filesystem structure looks like for the Obsidian app using the following:
# Download the appimage
wget https://github.com/obsidianmd/obsidian-releases/releases/download/v1.0.3/Obsidian-1.0.3.AppImage
# make it executable
chmod +x Obsidian-1.0.3.AppImage
# Run in the background
./Obsidian-1.0.3.AppImage &
# Look at the mount point
tree /tmp/.mount_Obsid*
